随着方法的发展，反转主要分为两个步骤。第一步是图像嵌入，其中编码器或优化过程嵌入图像以获取相应的潜在代码。之后，第二步旨在完善反转和编辑结果，我们将其命名为“结果”。尽管第二步显着提高了忠诚度，但感知和编辑性几乎没有变化，深处取决于第一步中获得的反向潜在代码。因此，一个关键问题是在保留重建保真度的同时获得更好的感知和编辑性的潜在代码。在这项工作中，我们首先指出，这两个特征与合成分布的逆代码的对齐程度（或不对准）有关。然后，我们提出了潜在空间比对反转范式（LSAP），该范式由评估度量和解决方案组成。具体来说，我们引入了归一化样式空间（$ \ Mathcal {s^n} $ space）和$ \ Mathcal {s^n} $ cosine距离（SNCD）以测量反转方法的不对准。由于我们提出的SNCD是可区分的，因此可以在基于编码器和基于优化的嵌入方法中进行优化，以执行均匀的解决方案。在各个域中进行的广泛实验表明，SNCD有效地反映了感知和编辑性，并且我们的对齐范式在两个步骤中都归档了最新的。代码可在https://github.com/caopulan/ganinverter上找到。

现有的文本识别方法通常需要大规模培训数据。由于缺乏带注释的真实图像，他们中的大多数依靠合成训练数据。但是，合成数据和真实数据之间存在域差距，这限制了文本识别模型的性能。最近的自我监督文本识别方法试图通过引入对比度学习来利用未标记的真实图像，这主要学习文本图像的歧视。受到人类学会通过阅读和写作识别文本的观察的启发，我们建议通过在我们的自我监督方法中整合对比度学习和掩盖图像建模来学习歧视和产生。采用对比学习分支来学习对文本图像的歧视，这模仿了人类的阅读行为。同时，首先引入了蒙版的图像建模，以了解文本识别，以了解文本图像的上下文生成，这类似于写作行为。实验结果表明，在不规则场景文本识别数据集上，我们的方法比以前的自我监督文本识别方法优于先前的自我监督文本识别方法。此外，我们提出的文本识别器超过了先前的最新文本识别方法，在11个基准测试中，平均5.3％，模型大小相似。我们还证明，我们的预培训模型可以轻松地应用于具有明显性能增益的其他文本相关任务。

基于深度学习的人网格重建方法具有构建更大网络的趋势，以实现更高的准确性。尽管是人网格重建模型的实际使用的关键特征，但往往忽略了计算复杂性和模型大小（例如，虚拟试用系统）。在本文中，我们呈现GTR，这是一种基于轻量级的姿势的方法，可以从2D人类姿势重建人网。我们提出了一种姿势分析模块，它使用曲线图形是利用结构化和隐式的关节相关性，以及将提取的姿势特征与网格模板组合以重建最终人体网格的网格回归模块。我们通过对人类3.6M和3DPW数据集进行广泛的评估，展示了GTR的效率和泛化。特别是，GTRS比SOTA姿势的方法POSE2MESH实现了更好的精度，同时仅使用10.2％的参数（PARAMS）和2.5％的跨越式3DPW数据集。代码将公开。

对抗性扰动对于证明深度学习模型的鲁棒性至关重要。通用的对抗扰动（UAP）可以同时攻击多个图像，因此提供了更统一的威胁模型，从而避免了图像攻击算法。但是，当从不同的图像源绘制图像时（例如，具有不同的图像分辨率）时，现有的UAP生成器不发达。在图像来源的真实普遍性方面，我们将UAP生成的新颖看法是一个定制的几个实例，它利用双杆优化和学习优化的（L2O）技术（L2O）技术，以提高攻击成功率（ASR）（ASR） ）。我们首先考虑流行模型不可知的元学习（MAML）框架，以将UAP生成器元素进行。但是，我们看到MAML框架并未直接提供跨图像源的通用攻击，从而要求我们将其与L2O的另一个元学习框架集成在一起。元学习UAP发电机（i）的最终方案的性能（ASR高50％）比预计梯度下降等基线的方案（II）比香草L2O和MAML框架的性能更好（37％）（当适用），（iii）能够同时处理不同受害者模型和图像数据源的UAP生成。

Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.

Despite recent progress towards scaling up multimodal vision-language models, these models are still known to struggle on compositional generalization benchmarks such as Winoground. We find that a critical component lacking from current vision-language models is relation-level alignment: the ability to match directional semantic relations in text (e.g., "mug in grass") with spatial relationships in the image (e.g., the position of the mug relative to the grass). To tackle this problem, we show that relation alignment can be enforced by encouraging the directed language attention from 'mug' to 'grass' (capturing the semantic relation 'in') to match the directed visual attention from the mug to the grass. Tokens and their corresponding objects are softly identified using the cross-modal attention. We prove that this notion of soft relation alignment is equivalent to enforcing congruence between vision and language attention matrices under a 'change of basis' provided by the cross-modal attention matrix. Intuitively, our approach projects visual attention into the language attention space to calculate its divergence from the actual language attention, and vice versa. We apply our Cross-modal Attention Congruence Regularization (CACR) loss to UNITER and improve on the state-of-the-art approach to Winoground.

During the deployment of deep neural networks (DNNs) on edge devices, many research efforts are devoted to the limited hardware resource. However, little attention is paid to the influence of dynamic power management. As edge devices typically only have a budget of energy with batteries (rather than almost unlimited energy support on servers or workstations), their dynamic power management often changes the execution frequency as in the widely-used dynamic voltage and frequency scaling (DVFS) technique. This leads to highly unstable inference speed performance, especially for computation-intensive DNN models, which can harm user experience and waste hardware resources. We firstly identify this problem and then propose All-in-One, a highly representative pruning framework to work with dynamic power management using DVFS. The framework can use only one set of model weights and soft masks (together with other auxiliary parameters of negligible storage) to represent multiple models of various pruning ratios. By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we are able to achieve stable inference speed, i.e., keeping the difference in speed performance under various execution frequencies as small as possible. Our experiments demonstrate that our method not only achieves high accuracy for multiple models of different pruning ratios, but also reduces their variance of inference latency for various frequencies, with minimal memory consumption of only one model and one soft mask.

How to effectively leverage the plentiful existing datasets to train a robust and high-performance model is of great significance for many practical applications. However, a model trained on a naive merge of different datasets tends to obtain poor performance due to annotation conflicts and domain divergence.In this paper, we attempt to train a unified model that is expected to perform well across domains on several popularity segmentation datasets.We conduct a detailed analysis of the impact on model generalization from three aspects of data augmentation, training strategies, and model capacity.Based on the analysis, we propose a robust solution that is able to improve model generalization across domains.Our solution ranks 2nd on RVC 2022 semantic segmentation task, with a dataset only 1/3 size of the 1st model used.

Clustering analysis of sequence data continues to address many applications in engineering design, aided with the rapid growth of machine learning in applied science. This paper presents an unsupervised machine learning algorithm to extract defining characteristics of earthquake ground-motion records, also called latent features, to aid in ground-motion clustering and selection. In this context, a latent feature is a low dimensional machine-discovered spectral characteristic learned through nonlinear relationships of a neural network autoencoder. Clustering can be performed on the latent features and used to select a representative archetypal subgroup from a large ground-motion suite. The objective of efficient ground-motion selection is to choose records representative of what the structure will probabilistically experience in its lifetime. Three examples are presented to validate this approach, including a synthetic spectral dataset and spectra from field recorded ground-motion records. Deep embedding clustering of ground motion spectra improves on the results of static feature extraction, utilizing characteristics that represent the sparse spectral content of ground motions.

Accelerated MRI aims to find a pair of samplers and reconstructors to reduce acquisition time while maintaining the reconstruction quality. Most of the existing works focus on finding either sparse samplers with a fixed reconstructor or finding reconstructors with a fixed sampler. Recently, people have begun to consider learning samplers and reconstructors jointly. In this paper, we propose an alternating training framework for finding a good pair of samplers and reconstructors via deep reinforcement learning (RL). In particular, we propose a novel sparse-reward Partially Observed Markov Decision Process (POMDP) to formulate the MRI sampling trajectory. Compared to the existing works that utilize dense-reward POMDPs, the proposed sparse-reward POMDP is more computationally efficient and has a provable advantage over dense-reward POMDPs. We evaluate our method on fastMRI, a public benchmark MRI dataset, and it achieves state-of-the-art reconstruction performances.